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Abstract 

Mixture models have been widely used in modeling of continuous 
observations. For the possibility to estimate the parameters of a mix- 
ture model consistently on the basis of observations from the mixture, 
identifiability is a necessary condition. In this study, we give some 
results on the identifiability of multivariate logistic mixture models. 

1 Introduction 

Mixture models have provided a mathematical-based approach to the 
statistical modeling of data generated from some underlying source. 
Mixture models based probability density function (pdf) have been 
used successfully in a number of applications ranging from speaker 
recognition [1] to bioinformatics [2j. Comprehensive surveys of esti- 
mation techniques, discussions, and applications are available in the 
work by [31 Eld EE]. 

The mixture of logistic distributions provides a more robust model 
to the fitting of normal mixture models, as observations that atypical 
of a component are given reduced weight in the calculation of its pa- 
rameters, for logistic distribution has similar shape with the normal 
distribution but has heavier tails and high kurtosis as compared to 
the exponentially decaying tails of a Gaussian. 

Malik and Abraham [10] suggested two families of multivariate 
logistic distribution which are multivariate analogue of the single- 
dimensional logistic distribution. In this paper, we prove some prop- 
erties on the identifiability of multivariate logistic mixtures. 
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Identifiability questions of mixtures must be settled before one can 
meaningfully discuss the problems of parameter estimation of a mix- 
ture model consistently on the basis of observations from the mixture. 
Identifiability makes sure that the correspondence between distribu- 
tions and mixture parameter vectors is one to one. Lack of identifia- 
bility is a serious problem if we intend to classify future observations 
into one of several classes from the component distributions. Identi- 
fiability of mixtures has been discussed by several authors, including 
Teicher [IT], Yakowitz and Spragins [12], AL-Hussaini and Ahmad [13], 
Ahmad and AL-Hussaini [14] . Ahmad [15] . and Holzmann et al. [16] . 



2 Main results 



2.1 Multivariate logistic mixture models 

[10] suggest two families of multivariate logistic distributions (MLD). 
We build mixture models on the first family. [10] define the joint dis- 
tribution function or cumulative distribution function (cdf) of X = 
(Xi,...,X p ) T for MLD as 

L(xi, ...,x p ) = [1 + Y^ k=x ex P(-^fc)] _i > (!) 
and the density function as 

l{x\ , . . . , Xp) 

= p!exp(- Y! k=l + ELi ex P("^)r (P+1) ' ( ^ 

where — oo < x\~ < oo, k = 1, 2, ...,p. In order to build mixtures based 
on MLD, we introduce center [i and scale X! into the distribution and 
then the cdf turned into 

L(xi,....,x p ;/x,E) = [1 + Y] P exp{-(x k - fi k )/a k )}-\ (3) 

and the density function turned into 

l{x 1 ,....,x p ;n,^2) 

1 p!exp(-ELi (x k -» k )/* k ) ( 4 ) 
L*=i " ; [i + Yi =l exp(-(x fc - m *)/<t*)](p+i) ' 

where \i = /i 2 , and scale XI = (a 1 , cr 2 , a p ). Here, we use 
the fact that 



(n 
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Let s be the number of mixture components. Let X = (X±, ...,X p ) T be 
a vector valued random variable distributed according to the following 
distribution function: 

G(X;rj) =V S mUXM,^), (5) 

where 77 = (7r,/i,£), 7r = (7Ti, 7r 2 , 7r s ), < n < 1, ]Ci=i ^ = 1- 
For i = 1,2, ...,s let /i = (fj,\, fx s ), £ = (£i,...,£ s ), and /jj = 
(pj,...,fi P ) G i? p , Sj = (Uj 1 , cr^) € (0,oo) p . Then the parameter 
space is denoted by V = (0, l) s x x ((0, oo) p ) s . L(«; Mi, £») is the 

cdf of the multivariate logistic (//j, Sj)-distribution. 

2.2 Identifiability 

Identifiability is a necessary condition for the possibility to estimate 
the parameters of a mixture model consistently on the basis of obser- 
vations from the mixture. It makes sure that the correspondence be- 
tween distributions and mixture parameter vectors is one to one. Let 
£ = {£(X; fi, S) : (/j, ~S) G 0} constitute the family of p-dimensional 
logistic distributions indexed by a parameter from the parameter space 
0, where = BP x (0,oo) p . Let G G SK be a cdf on 0. Then 
H(X.,G) = f & L(X; fi, ~S)dG(fi, X) defines a mixture distribution. 
The distribution is called the G-mixture of £. In this work we are 
dealing with finite mixtures, that is to say 9\ being the set of distribu- 
tions on with finite support. Consequently, we write G{(jl, X) for the 
mixture proportion corresponding to /jj, Sj). If X is fixed, then 
£ becomes £ s = {L s (X;/j) : (//,£) G S }, where S = R p x X. 
Respectively let D^s being the set of distributions on 0£ with finite 
support, and then H(X., Gs) = J@ Ls(X; /i)dGs(/i) defines a mixture 
distribution on £s, where Gs G Following Yakowitz and Spra- 
gins |12| . the mixture model generated by the family £ with mixing 
distribution in D\ is said identifiable if for Gi,G 2 G : H(»,Gi) = 
H(;G 2 )^G 1 = G 2 . 

Theorem 1. When p = 1, the class £ with parameter set and 
mixing distribution in 9\ as defined above is identifiable. 

Theorem 2. When p > 1, the class £s with parameter set 0s and 
mixing distribution in £Rs is identifiable. 

The usual approaches based on moment generating function, char- 
acteristic function and Laplace transforms [11| \12\ [13] can not be ap- 
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plied to this situation. Individual methods are needed. We first prove 
two lemmas which will be used in the proof of our theorems. 

Lemma 1. Assume that 

V S di[l + exp(-(x - = 0, (6) 

«/ (//j, dj), i = 1, s are distinct and Oi > 0, i = 1, s, i/ien dj = ; 
i = 1, ...,s. 

Proof. We proof this lemma by contradiction. Assume that d% ^ 0, i = 

s 

l,...,s. Let x — > oo in Eqn. ([6|), we obtain ^ di = 0. Thus with 

i=l 

Eqn. ([6]), we have 

V S di[l + exp((x - Mi)M)] -1 = 0. (7) 

* *2=1 

If now o\ = ... = o~ s , then in,i = l,...,s are distinct. Multiply both 
sides of Eqn. (J7J) by l + exp((x — /ii)/o"i) and take the derivative w.r.t. 
x, we obtain 

y\ o fij[exp(-/ii/cri) - exp(-^ i /cr l )][l + exp((x - ^)/crj)]" 2 
^*= 2 (8) 

= 0. 

Then iteratively multiply both sides of Eqn. (|HJ) by 1 + exp((x — 
w) / > * = 2, s and derive respect to x similar to the process ap- 
plied to Eqn. ([7J) above, we have d s (Ili=i [ ex P( — t^i/^i) ~ ex P( — Mi/°"i)])[l" 
exp( a: ~ Ats = 0. Then we obtain d s = which contradicts to the 
assumption. 

On the other hand, without loss of generality, assume o~\ = 02 = 

... = <7fc < <7fc + i < ... < a s . Multiply both sides of Eqn. ([7]) by 

k 

exp(x/<r), where a < a±, and let x — > 00, we have ^ <ij exp(//j/<7j) = 0. 

i=l 

Derive both sides of Eqn. ([7]) with respect to x, we have 

V S _ (diMfl + exp((x - Hi)/oi)\- x 
-dj/o-jfl + exp((x - /ij)/o-j)] -2 ) = 0. 

Multiply both sides by exp(x/V), where a < <7i/2, and let x — > 00, 

fe 

we have ^ dj exp(2/ij/<Tj) = 0. Then similarly, derive j times with 
i=i 
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respect to x, and multiply by exp(x/cr), where a < = 2, 

k 

Then let x — > oo, we obtain Yl <k ex P0 A*i / ^i) = 0. Thus we obtain 

i=l 



£ <kexp(m/(7i) = 
i=l 
fe 

^2 di exp(2^/f7j) = 

4 = 1 



X] <kexp(k(ii/<Ti 
i=i 



(10) 



Prom Vandermonde's rule, we have dj = 0, z = 1, 
to the hypothesis. Thus the lemma is proved. 



, k which contradict 
□ 



Lemma 2. Assume that 



M 1 + Y] P , , exp(-(x fe 



Mi)/ "* 1 )] -1 = 0, 



(11) 



« = l,...,s are distinct, where fn = (nj, fj%), m € R P ,Y± = 
(a , <r p ), and 5] € (0, oo) p £/ien dj = 0, i = 1, s. 

Proof. We give a proof by inductive on s and p. When s = 1, it is 
trivial and p = 1, it is Lemma HJ Setting s > 1 fixed, let p > 1 and 
assume p — 1 is true. Let x\ = a l x — y\,X2 = cr 2 x — y2, then we can find 
(2/1,2/°) satisfies that if (n},v%) + (m),/^), then exp((y? + nD/a 1 ) + 
exp((y0 + /i2)/ c7 2 ) /exp((yO + ^))/ ( 7 1 ) + exp((yO + / u])/a 2 ). Otherwise, 
for each (yi, 2/2)1 there exists a pair that(^,/i 2 ) 7^ and 
exp ( (yi + /4 ) /a 1 ) + exp ( [y 2 + fi 2 ) /a 2 ) = exp ( (y 1 + fi) ) /a 1 ) + exp ( (y 2 + 
/u,J)/a 2 ). Then there must exists a pair and a two dimensional 



/ h,k h\ 

series (y^ ,y%) 



where lim y^ 



00, and for fixed h, lim y'[ 



h.k 



00, 



that (jj,},h?) # O^juf)' and 



exp((^ fc + ^ 1 )/ ( t 1 ) + exp((y£ + M ?)/a 2 
exp((^ fe + ^)/a l ) + exp((y£ + $)/o 2 \ 



(12) 



00 in Eqn. [T21 we obtain fj,j = p,j. Then let 
// 2 , which contradict to the assumptions. 



Let h fixed, and k 
h — > 00, we have /if 
So let x\ = o x x 
to the situation p — 1. Thus the lemma is proved. 



o" 2 x — 2/2 m Eqn. [TTJ then it is reduced 



□ 
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Proof of Theorem [TJ 



Proof. According to the theorem of [12] , Theorem [T] is equivalent to 
the linear independence of £ over the field of real numbers which can 
be inferred immediately from Lemma [21 Thus when p = 1 the class £ 
with parameter set O and mixing distribution in 9\ as defined above 
is identifiable. □ 

Proof of Theorem [2j 

Proof. Similar with the proof above, from Lemma [2] and the theorem 
of [T2], Theorem [2] is obvious. □ 

Open Problem: When p > 1, the class £ with parameter set 
and mixing distribution in 91 is identifiable. 



3 Conclusions 

In this work, we examine the identifiability property of logistic mixture 
models. We first proof that the univariate logistic mixture models is 
identifiable. Then we also proof the identifiability for a special case 
of the multivariate logistic mixture models. At last we propose a 
conjecture about identifiability of the multivariate logistic mixture 
models. 
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